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ABSTRACT 

ELECT  BiLAT  is  a  prototype  game-based  simulation 
for  Soldiers  to  practice  conducting  bilateral  engagements 
in  a  cultural  context.  The  prototype  provides  students 
with  the  experience  of  preparing  for  a  meeting  including 
familiarization  with  the  cultural  context,  gathering 
intelligence,  conducting  a  meeting  and  negotiating  when 
possible,  and  following  up  on  meeting  agreements  as 
appropriate.  The  ELECT  BiLAT  architecture  is  based  on 
a  commercial  game  engine  that  is  integrated  with  research 
technologies  to  enable  the  use  of  virtual  human 
characters,  scenario  customization,  as  well  as  coaching, 
feedback  and  tutoring. 

Because  the  prototype  application  is  intended  to  be  a 
learning  environment,  pedagogy  has  been  central 
throughout  development.  The  project  followed  a  five- 
phase  process:  (1)  analyze  the  training  domain;  (2) 
develop  a  story  board  prototype;  (3)  implement  a 
computer  version  of  the  training  prototype;  (4)  refine 
training  objectives  and  link  their  conditions  and  standards 
to  game  activities;  and  (5)  develop  training  support 
content  for  students,  instructors,  and  training  developers. 
The  goal  is  an  authorable  game-based  environment  that 
uses  the  pedagogy  of  guided  discovery  for  training 
Soldiers  in  the  conduct  of  bilateral  engagements  within  a 
specific  cultural  context. 

1.  INTRODUCTION 


development  of  the  ELECT  BiLAT  training  prototype, 
one  of  the  first  products  of  the  Learning  with  Adaptive 
Simulation  and  Training  (LAST)  Army  Technology 
Objective  (ATO)  program.  The  purpose  of  the  LAST 
ATO  is  to  develop  tools  and  methods  to  prepare  leaders 
and  Soldiers  for  conducting  operations  against  an 
asymmetric  enemy  in  the  Global  War  on  Terrorism. 

The  ELECT  BiLAT  prototype  is  a  game-based 
simulation  that  provides  Soldiers  a  practice  environment 
for  conducting  bilateral  meetings  and  negotiations  in  a 
cultural  context.  While  it  is  too  early  to  report  on  the 
effectiveness  of  the  prototype  as  a  training  tool,  there  are 
a  number  of  aspects  of  this  project  that  are  worth 
reporting  to  the  technology  and  training  development 
communities.  In  particular,  ELECT  BiLAT:  (1)  addresses 
a  non-kinetic  training  domain  that  is  relevant  to  the 
Contemporary  Operating  Environment  (COE),  (2) 
employs  customized  game-play  mechanics  to  provide  a 
more  immersive  and  interactive  experience,  (3)  makes 
extensive  use  of  story-based  scenarios,  (4)  leverages 
virtual  human  and  game  technologies  to  support  social 
interaction,  (5)  incorporates  intelligent  tutoring  to 
enhance  the  learning  experience,  (6)  was  designed  to 
enable  rapid  scenario  development  and  modification  (i.e., 
the  scenarios  are  authorable  by  the  end-users),  and  (7) 
used  a  pedagogically-oriented,  five-phase  approach 
throughout  the  development  of  the  training  prototype. 

While  the  five-phase  approach  described  below 
reflects  many  of  the  best  practices  of  the  training 
development  community,  what  was  special  in  this  case 
was  that  it  was  applied  across  the  processes  of  a  multi¬ 
disciplinary  enterprise  comprising  technologists,  game 


This  paper  describes  the  underlying  processes  used 
during,  and  some  of  the  initial  lessons  learned  from,  the 
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designers  and  industrial  game  developers,  creative 
writers,  psychologists  and  Soldiers.  The  LAST  ATO 
partners  include  the  Army’s  Research,  Development  and 
Engineering  Command’s  Simulation  and  Training 
Technology  Center  (RDECOM  STTC),  the  University  of 
Southern  California’s  Institute  for  Creative  Technologies 
(USC-ICT),  the  Army  Research  Laboratory  Human 
Research  &  Engineering  Directorate  (ARL  HRED),  and 
the  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI).  This  team  worked  collaboratively  to 
develop  a  game-based  training  prototype  that  leverages 
findings  from  the  science  of  learning  and  provides  a 
sustainable  pedagogical  structure  by  using  training 
objectives  as  a  focal  point. 

Whereas  there  are  many  examples  of  commercial 
games  that  have  been  adapted  to  support  military  training 
objectives,  ELECT  BiLAT  is  a  game-based  training 
application  built  from  the  ground  up  with  specific  end- 
state  training  objectives  in  mind.  The  pedagogical  design 
section  describes  an  explicit  framework  for  developing 
training  objectives  and  refining  them  as  the  training 
application  is  built.  The  goal  of  this  paper  is  to  describe 
the  framework  so  that  other  developers  can  incorporate  it 
into  their  future  efforts  to  produce  pedagogically  sound, 
interactive,  virtual  training  systems.  In  addition,  this  paper 
reports  on  some  of  the  lessons  learned  in  the  course  of 
developing  this  tool. 

2.  BILATERAL  ENGAGEMENT 

The  purpose  of  the  ELECT  BiLAT  training  prototype 
is  to  provide  Soldiers  with  an  interactive,  game-like 
simulation  for  practicing  their  skills  in  conducting 
meetings  and  negotiations  with  local  leaders  in  a  specific 
cultural  context.  For  ELECT  BiLAT,  the  training  domain 
of  bilateral  meetings  was  chosen  for  its  importance  in 
current  and  future  stability,  security,  transition,  and 
reconstruction  operations  (Department  of  Defense,  2005; 
Chiarelli  &  Michaelis,  2005). 

The  term  ‘bilateral  engagement’  is  used  to  describe 
the  intentional  activities  of  discussion,  conference  and 
negotiation  that  take  place  between  two  parties  to  bring 
about  agreement  (Wunderle,  2005b).  In  a  military 
context,  bilateral  engagements  occur  at  all  levels  of 
command,  from  squad  leader  to  the  general  officer  level; 
it  is  an  activity  that  must  be  integrated  with  other 
operations  in  order  to  yield  a  successful  campaign  in  a 
region.  Another  reason  for  the  importance  of  mastering 
this  skill  set  is  that  a  successful  bilateral  engagement  can 
save  lives  by  defusing  situations  within  a  town  or  region 
where  there  exists  the  potential  for  agreement  rather  than 
violence.  Lack  of  proficiency  in  this  domain  has  the 
potential  to  cause  second-  and  third-order  effects  with 
long-lasting  negative  consequences. 


Wunderle’s  (2005b)  approach  to  bilateral 
engagement  in  military  operations  was  one  of  the  most 
influential  sources  in  the  design  of  the  ELECT  BiLAT 
prototype.  Wunderle  draws  heavily  from  Ury  and  Fisher’s 
(1991)  work  on  principled  negotiation,  and  he  mapped 
these  principles  onto  the  military  decision-making 
process.  Wunderle  emphasizes  the  preparation  phases  of  a 
bilateral  engagement1  and  not  only  identifying  one’s  own 
intended  outcomes  but  also  identifying  and  anticipating 
the  objectives  of  the  meeting  partner.  This  analytical 
process  maps  to  two  of  Ury  and  Fisher’s  negotiation 
principles;  (1)  separate  the  people  from  the  problem  and 
(2)  focus  on  interests,  not  positions.  Successfully  applying 
these  principles  requires  extensive  research  to  identify  the 
problem  and  the  interests  of  the  partners. 

Following  the  initial  analysis  of  the  objectives  and 
interests  of  both  parties,  the  next  phase  of  Wunderle’s 
methodology  is  what  he  calls,  “develop  intended  outcome 
strategy,’’  which  maps  to  a  third  negotiation  principle 
from  Ury  and  Fisher  (1991) — invent  options  for  mutual 
gain.  It  is  during  this  phase  that  the  planner  needs  to 
identify  a  bottom  line  that  serves  as  an  acceptable 
alternative  outcome  to  the  intended  outcome.  While  this 
process  does  not  guarantee  that  the  planner  will  identify 
an  option  that  maximizes  mutual  gain,  higher 
headquarters  can  suggest  win-win  solutions  during  this 
phase  through  coaching  and  feedback. 

After  planning  is  complete,  the  meeting  is  conducted, 
with  particular  attention  being  paid  to  time  management 
and  sequencing.  While  the  planning  process  is  a  necessary 
condition  for  success,  the  ability  to  interact  at  a  personal 
level  is  also  crucial.  Again,  good  preparation  can  help, 
especially  if  it  includes  gaining  an  understanding  of  the 
cultural  background  of  the  meeting  partner.  Wunderle 
(2005a)  suggests  that  understanding  the  culture2  such  as 
communication  styles,  perception  (both  ways)  and  how 
respect  is  shown  (or  not)  profoundly  influences  the 
outcome  of  a  bilateral  engagement.  Ury  and  Fisher 
identify  these  same  factors  as  crucial  in  principled 
negotiation.  These  thoughts  were  also  echoed  by  the 
Officers  who  shared  their  personal  experiences  during 
interviews  for  this  project. 

Finally,  following  the  meeting,  it  is  critical  that  the 
leader  follow  through  on  promises  made,  identify 
outstanding  issues,  and  plan  next  steps.  In  many  cases,  the 
knowledge  gained  from  one  bilateral  engagement  may 
have  a  direct  bearing  on  who  the  next  meeting  partner 


1  Wunderle  equates  this  process  to  intelligence 
preparation  of  the  battlefield  (IPB,  Department  of  Army, 
1994). 

2  Wunderle  defines  culture  as  a  shared  set  of  traditions, 
belief  systems,  and  behaviors. 
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should  be  and  what  issues  are  raised  in  subsequent 
meetings. 

3.  GAME-ORIENTED  DESIGN 

USC-ICT  engaged  faculty  and  students  from  the 
Game  Innovation  Laboratory3  of  the  Cinema-Television 
School  at  USC  to  design  a  game-like  simulation  that 
would  implement  the  principles  and  processes  just 
described.  As  described  later,  ARL-HRED  and  ARI  ,  in 
collaboration  with  researchers  from  USC-ICT,  performed 
a  task  analysis  on  key  materials  that  detailed  the  steps  and 
considerations  in  the  process  of  conducting  bilateral 
negotiations.  The  result  was  a  set  of  training  objectives 
that  the  ELECT-BiLAT  prototype  needed  to  address.  The 
game  design  team  proceeded  to  create  a  story  board 
prototype  that  allowed  them  to  iteratively  test  game  play 
mechanics  before  committing  to  the  expensive  step  of 
writing  computer  code  and  creating  artwork.  The 
prototype  was  play-tested  internally  and  with  subject 
matter  experts.  In  each  phase  of  testing  the  feedback  was 
factored  into  the  design  of  another  version  of  the  system. 

In  the  end,  the  game  was  designed  to  focus  the 
student  on  performing  tasks  directly  related  to  the  training 
objectives  identified  during  the  task  analysis.  The  design 
supported  the  concept  of  a  multi-phase  process  for 
bilateral  engagement:  meeting  preparation,  rehearsal, 
conducting  the  meeting,  and  after  action  review.  The 
game  play  experience  depended  on  the  attention  to  details 
in  the  process  (e.g.,  the  modeling  of  constraints  like  time, 
resources,  and  trust).  For  example,  during  the  preparation 
phase,  each  information  resource  accessed  by  the  student 
is  associated  with  a  time  cost  associated.  However,  the 
game  does  not  have  to  model  actions  in  real-time.  Rather, 
time  is  deducted  from  an  in-game  clock  to  represent  how 
long  it  might  have  taken  in  the  real  world  to  track  down 
the  information  and  process  it.  During  the  meeting  phase, 
the  attitude  of  the  meeting  partner  is  affected  in  part  by 
the  level  of  trust  that  has  been  generated  by  earlier  actions 
and  things  said.  The  trust  variable  is  one  of  a  number  of 
variables  that  affect  the  game  play,  but  it  is  the  only  one 
exposed  to  the  student.  It  is  a  simplistic  representation  of 
a  complex  behavior  but  serves  as  a  way  of  reminding  the 
student  of  the  importance  of  building  trust  with  the 
meeting  partner. 

One  of  the  criticisms  of  game-based  training  has  been 
that  that  there  is  too  much  emphasis  on  graphics  and  the 
“wow!”  factor  at  the  expense  of  training  effectiveness 
(Hays,  2005).  This  need  not  be  the  case,  because  there  is 
some  empirical  evidence  on  how  to  develop  effective 
immersive  training  systems  (Ricci,  Salas,  and  Cannon- 


3  Including  Prof.  Scott  Fisher,  Prof.  Chris  Swain,  Prof. 
Peggy  Weil,  Jesse  Virgil  and  Jessica  Rosenblatt. 


Bowers,  1996).  Garris,  Ahlers,  and  Driskell  (2002) 
highlight  game  features  leading  to  better  training 
performance.  Campbell,  Quinkert,  and  Burnside  (2000) 
suggest  that  structuring  training  based  on  training 
objectives  leads  to  improved  training  outcomes. 
Belanich,  Sibley,  &  Orvis  (2004)  found  that  trainees  are 
more  likely  to  recall  game  content  if  the  “storyline”  is 
relevant  to  the  training  objectives.  Based  on  these  results, 
the  design  philosophy  behind  the  ELECT-BiLAT  training 
game  was  to  take  a  training-objective-centric  approach  to 
game  design  in  concert  with  game  designers. 

4.  STORY-BASED  SCENARIOS 

A  crucial  component  of  the  ELECT  BiLAT  prototype 
is  the  use  of  story-based  scenarios.  A  training  experience 
in  ELECT  BiLAT  does  not  end  with  a  single  bilateral 
meeting,  rather,  it  is  a  series  of  meetings  that  accomplish 
a  broader  set  of  mission  objectives.  This  aspect  of  the 
game  design  reinforces  the  notion  that  Army  leaders  are 
dealing  with  complex  social  networks  that  interconnect 
and  affect  one  another  in  sometimes  surprising  ways. 

The  scenarios  were  loosely  based  on  open-source 
stories  about  problems  encountered  in  different  locales 
around  the  world.  The  writers  developed  fictional 
characters  with  varying  backgrounds  and  attitudes  that 
require  the  player  to  understand  each  individual,  their 
interests  and  issues,  and  their  cultural  background  as  it 
relates  to  the  others  in  the  social  networks  of  the  scenario. 

There  is  a  qualitative  difference  between  a  story- 
based  scenario  and  the  event-based  scenarios  that  are 
typically  used  to  drive  military  simulations.  Stories  have 
rich  characters  with  the  power  to  engage  the  user  during 
interactions.  Good  stories  have  interesting  plots,  with 
dilemmas,  suspense,  and  unexpected  twists.  A  well 
designed  story  contains  many  links  among  characters  and 
events,  so  it  is  not  just  a  physics-based  cause-and-effect 
experience,  rather,  “social  physics”  are  at  play,  which  are 
much  more  unpredictable.  A  well-crafted  story-based 
scenario  allows  the  student  to  have  a  social  experience 
that  is  not  achievable  in  an  event-driven  simulator. 


5.  TECHNOLOGIES 

The  ELECT  BiLAT  prototype  is  structured  around 
the  major  phases  of  a  bilateral  engagement:  preparation, 
rehearsal,  meeting  and  after-action  review.  While  each 
phase  requires  specific  functionality,  they  all  share  the 
need  for  visualization  and  a  user  interface.  ICT’s 
Integrating  Architecture  (IA)  (van  Lent,  et  ah,  2004) — a 
communications  and  software  platform  supporting 
research  component  integration — provides  the  graphical 
environment  via  an  embedded  Unreal  3D  game  engine 
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commercially  available  through  Epic  Games.  To  support 
the  needs  of  the  game  play  and  scenario  content,  as  well 
as  the  goal  of  authorability,  the  system  architecture 
underneath  the  platform  was  designed  as  a  service- 
oriented,  heterogeneous  and  network-based  architecture 
consisting  of  discrete  agents  providing  the  following 
functions:  graphical  user  interface,  dialogue  management, 
social  simulation,  experience  management,  character 
animation,  online  coaching  and  reflective  tutoring. 

The  Experience  Manager  (EM)  supports  the 
preparation  phase  whereby  the  trainee  conducts  research 
and  rehearsals  prior  to  engaging  with  the  selected  meeting 
partner.  The  EM  manages  character  availability,  the 
research  available  on  each,  and  the  set  of  game  objectives 
requiring  completion  to  progress  in  the  scenario.  It  is  also 
responsible  for  initializing  the  game  environment  prior  to 
the  meeting  starting.  The  preparation  phase  is  very 
information-intensive,  and  required  several  usability 
studies  in  order  to  finalize  an  accessible  interface. 
Preparation  was  also  authoring-intensive  requiring 
scenario  details  that  could  be  discovered  and  then  linked 
to  executable  actions  in  the  meeting  to  follow. 

Meetings  in  ELECT  BiLAT  are  enabled  through  a 
simulation  that  requires  the  support  of  several 
components.  The  Dialogue  Manager  (DM)  handles  turn- 
by-turn  conversation  for  the  virtual  character  and  passes 
utterances  to  Smartbody  for  non-verbal  behavior 
generation.  Smartbody  sends  these  animation  and  speech 
timing  signals  to  IA  in  order  to  render  the  character’s 
onscreen  performance.  Throughout  the  meeting,  the  user 
is  assisted  by  the  coach  based  on  relevant  meeting  traffic. 
The  coach  relies  on  the  tutor  system’s  knowledge 
database  that  is  populated  with  pedagogical  meta¬ 
information  for  the  entire  meeting.  Finally,  the  Psychsim 
social  simulation  determines  negotiation  decisions  during 
the  sub-phase  of  the  meeting  where  business  terms  are 
being  offered  and  requested. 

Authoring  scenario  details  for  the  meeting  consisted 
of  building  high-level  meeting  actions  (e.g.,  flatter  host) 
and  specifying  the  possible  effects  on  the  meeting  partner. 
Dialogue  utterances  were  written  and  then  mapped  in  the 
system  to  an  effect  on  the  meeting  partner.  They  could 
also  be  mapped  to  spontaneous  conversational 
interjections  triggered  by  defined  pre-conditions  (e.g.,  the 
meeting  partner  offering  tea  when  pleased  with  how  the 
meeting  is  progressing). 

All  meeting  actions  were  linked  to  training  objectives 
that  are  tracked  by  the  coach  and  tutor  system.  Dialogue 
utterances  are  then  processed  through  a  non-verbal 
generation  and  speech  workflow  to  cache  the  Smartbody 
animation  data  needed.  Finally,  the  more  strategic 
negotiation  actions  are  defined  and  parameterized  with 


respect  to  meeting  partner  goals  for  consideration  by  the 
PsychSim  system. 

To  complete  the  game  cycle,  an  after-action  review 
follows  all  meeting  engagements  and  is  hosted  by  the 
reflective  tutor.  During  a  simulated  meeting,  the  tutor 
dynamically  builds  an  agenda  to  review  with  the  trainee. 
The  post-meeting  walkthrough  consists  of  a  Socratic 
discussion  of  both  positive  and  negative  meeting  events. 
Visualization  was  supported  by  the  Dialogue  Manager’s 
utterance  history  and  a  “VCR”  playback  via  Smartbody  in 
a  virtual  video  display.  At  the  conclusion  of  the  analysis 
session  and  the  meeting  cycle,  the  trainee  returns  to  the 
objectives  status  screen  in  order  to  consider  what  to  do 
next  based  upon  what  s/he  has  learned. 

The  software  infrastructure  supports  the  overall 
training  objectives  of  the  game  as  well  as  the  scenario 
content  needed  to  provide  an  immersive  student 
experience.  Some  of  those  components  were  the  result  of 
the  transition  of  research  technologies,  many  of  which 
were  never  previously  available  for  game  environments. 

The  SmartBody  project  (Lee  &  Marsella,  2006; 
Kallmann  &  Marsella,  2005)  is  part  of  the  ICT’s  virtual 
human  project  (Swartout  et  al,  2006).  SmartBody  was 
designed  to  employ  a  range  of  different  animation 
techniques.  Most  significantly,  it  supports  an  approach  to 
procedurally  generated  animation  that  generates  behavior 
dynamically  and  automatically,  given  an  utterance.  The 
integration  of  this  technology  makes  it  possible  to  more 
rapidly  generate  and  modify  scenarios  beyond  what  is 
currently  contained  within  the  system. 

PsychSim  (Marsella  &  Pynadath,  2004;  Pynadath  & 
Marsella,  2005)  is  an  AI  framework  for  implementing 
social  simulation  and  provides  a  great  deal  of  power  to 
model  the  impact  that  group  membership  has  on  the 
attitudes  of  a  meeting  partner,  and  vice  versa.  To  facilitate 
the  authoring  of  PsychSim  models,  a  tool  was  developed 
that  takes  an  author-provided  high-level  specification  of 
the  negotiation  participants  and  automatically  translates 
that  specification  into  PsychSim  models.  PsychSim  also 
provides  explanation  facilities  for  its  behavior  that  will 
eventually  be  coordinated  with  the  explanation  and 
dialogue  capabilities  of  our  Explainable  AI  (XAI)  system. 

6.  COACHING  AND  REFLECTIVE  TUTORING 

For  learning  to  be  effective  it  should  be  guided 
(Kirschner,  et.  al.,  2006).  To  provide  guidance  in  ELECT 
BiLAT,  an  intelligent  tutoring  system  (ITS)  is  included  as 
a  key  component  of  the  system.  Building  off  previous 
work  (Lane,  et.  al.,  2006),  two  kinds  of  ITS  technology 
were  implemented;  a  coach  and  a  reflective  tutor.  The 
coach  is  used  during  meetings  to  provide  feedback  and 
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hints,  while  the  reflective  tutor  works  with  the  coach  to 
guide  after-action  reviews  (AARs). 

The  coach  runs  in  the  background  watching  every 
action  taken  by  the  player  during  meetings.  Each  action 
is  assessed  as  correct,  incorrect,  or  mixed.  To  make  this 
determination,  it  consults  an  expert  model  (also  part  of  the 
ITS  architecture)  that  looks  up  the  learning  objective(s) 
associated  with  the  action  and  whether  or  not  that  action 
moved  the  negotiation  partner  closer  to  an  ideal  state 
(e.g.,  when  trying  to  build  trust,  did  the  action  actually 
improve  trust?).  The  coach  also  decides  whether  or  not  to 
give  explicit  feedback  after  each  action  by  consulting  pre¬ 
configured  settings.  For  example,  an  instructor  can  adjust 
the  coach  to  give  only  negative  feedback  after  errors.  Or, 
the  coach  can  be  set  to  give  positive  feedback  on  a 
schedule  (e.g.,  every  second  correct  action),  or  in  some 
combination  with  negative  feedback.  Finally,  the  coach 
maintains  a  rudimentary  model  of  the  learner  based  on 
learning  objectives,  and  can  give  targeted  feedback  when 
certain  learning  objectives  are  active. 

Since  there  is  little  time  for  extended  periods  of 
discussion  during  a  meeting,  effort  was  taken  to  keep 
coaching  utterances  short  and  to  the  point.  Using  the 
coach’s  assessment,  the  reflective  tutor  generates  an 
agenda  of  topics  to  discuss  during  the  AAR.  The  tutor 
then  uses  a  cognitive  model  that  includes  a  variety  of 
tutoring  tactics  to  address  these  topics.  With  such  tactics 
as  direct  feedback,  conceptual  questioning,  “what  else” 
questions  (asking  about  alternative  courses  of  action),  and 
XAI  investigations  (allowing  students  to  ask  virtual 
humans  in  the  game  to  explain  their  actions),  the 
reflective  tutor  is  able  to  go  beyond  simple  mission 
statistics  and  discuss  the  conceptual  issues  of  the  domain. 

7.  PEDAGOGICALLY  ORIENTED 
DEVELOPMENT  PROCESS 

The  development  of  ELECT  BiLAT  started  with  a 
critical  assumption:  the  game  environment  is  not  a  vehicle 
by  which  learning  is  delivered,  but  rather  it  provides  a 
practice  environment  to  augment  and  internalize  lessons 
learned.  Prior  to  interacting  with  the  ELECT  BiLAT 
training  prototype  the  student  should  receive  instruction 
on  how  to  conduct  a  meeting  engagement,  how  to 
negotiate,  and  how  the  particular  culture  being  studied 
will  influence  the  conduct  of  meetings  and  negotiations. 

One  lesson  learned  thus  far  is  there  are  at  least  five 
phases  in  the  process  of  developing  an  immersive  training 
system  that  is  pedagogically-structured  and  designed  so 
that  new  scenarios  can  be  authored.  The  phases  of  the 
development  process  identified  here  are:  (1)  analyze  the 
training  domain,  (2)  develop  a  story  board  prototype,  (3) 
implement  a  computer  prototype,  (4)  further  specify  and 


refine  training  objectives,  conditions  and  standards  and, 
(5)  develop  training  support  material.  While  this  process 
was  used  specifically  for  the  ELECT-BiLAT  training 
game,  it  could  also  be  used  by  other  training  system 
developers  who  are  interested  in  producing  pedagogically 
structured,  immersive  training  environments. 

1.  Analyzing  the  Training  Domain.  The  first  stage  in 
the  rapid  development  of  this  training  application  began 
with  examining  the  Contemporary  Operating 
Environment  (COE)  to  determine  the  focus  of  training  for 
the  application.  The  next  step  was  to  organize  the 
knowledge  and  skill  domain  of  what  would  be  trained. 
However,  there  was  no  prior  official  delineation  of  this 
training  domain,  which  clearly  represents  a  “wicked 
problem,’’  with  better  or  worse  rather  than  right  or  wrong 
solutions  (Rittel  &  Weber,  1973).  Therefore,  this  required 
the  development  of  training  objectives  based  on  a  task 
analysis  of  the  domain. 

Project  team  members  interviewed  subject  matter 
experts  (SMEs)  and  reviewed  available  literature  in  the 
training  domain  as  part  of  the  cognitive  task  analysis 
process  (DuBois  &  Shalin,  2000;  Hackos  &  Redish, 
1998).  This  process  of  identifying  and  organizing 
training  domain  content  started  with  discussions  and 
interviews  in  conjunction  with  the  initial  Army  customer, 
but  also  included  other  interviews  at  various  military 
facilities  with  related  domain  experience.  SMEs  were 
asked  about  appropriate  and  inappropriate  actions,  the 
conditional  variables  that  influence  specific  courses  of 
actions,  and  how  they  currently  instruct  trainees.  SME 
interview  data  was  combined  with  available 
documentation  specifying  the  necessary  phases,  tasks  and 
key  personnel.  The  documentation  used  as  primary 
references  for  this  information  included  field  manuals 
(Air  Land  Sea  Application  Center,  2004;  Department  of 
Army  1993,  1994),  articles  written  by  deployed  military 
personnel  and  leaders  (Heidecker  &  Sowards,  n.d.,  2004; 
Karabaich,  2005),  and  research  reports  (Meliza,  1996; 
Morrison  &  Meliza,  1999). 

These  data  were  then  coalesced  into  training 
objectives  that  indicate  appropriate  and  inappropriate 
tasks,  based  on  specific  conditions,  and  the  standard  to 
which  these  tasks  should  be  demonstrated;  a  format 
familiar  to  Army  instructors  (Department  of  the  Army, 
2003).  Each  training  objective  consisted  of  three  sections; 
the  general  description  of  the  training  objective,  the 
conditions  where  the  training  objective  was  relevant,  and 
the  standards  indicating  the  actions  that  would 
demonstrate  adherence  to  the  training  objective. 

One  training  objective  identified  was  the  use  of  a 
win/win  strategy  during  negotiation  (described  in  section 
2).  During  the  training  domain  analysis  process,  LTC 
Wunderle  and  three  other  Army  SMEs  emphasized  the 
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importance  of  win/win.  To  illustrate  the  role  pedagogy 
played  throughout  the  development  process,  the  impact  of 
this  training  objective  on  the  development  of  the  training 
tool  will  be  described  in  the  subsequent  steps. 

2.  Develop  Story  Board  Prototype.  As  described  in 
section  3,  the  game  designers  developed  a  paper  and 
pencil  prototype,  or  story  board.  This  prototype  game 
design  was  linked  to  the  results  of  the  analysis  phase.  An 
instructional  planning  document  based  on  the  task 
analysis  and  the  training  objectives  was  developed  to 
delineate  the  training  domain  information  and  outline  the 
requirements  of  the  game.  Pedagogical  controls  were  also 
drafted  (e.g.,  how  feedback  could  be  structured),  and 
implemented  in  the  paper  board-game. 

The  prototype  was  demonstrated  for  instructors  that 
would  ultimately  use  the  training  system  and  additional 
Army  personnel.  The  feedback  gained  from  these 
demonstrations  was  used  to  guide  modifications  to  the 
training  game  before  subsequent  demonstrations  in  an 
iterative  development  process.  The  use  of  a  paper 
prototype  allowed  for  rapid  modifications  to  the  prototype 
without  incurring  expensive  and  time  consuming 
computer  programming  resources.  It  was  also  during  this 
phase  that  in-game  content  started  to  be  created  based  on 
SME  vignettes.  The  content  was  tied  to  the  training 
objectives  and  validated  by  SMEs.  The  training  objectives 
were  used  as  an  information  source  for  game  content 
(characters,  stories,  etc.)  and  the  training  objectives 
served  as  an  overarching  framework. 

For  example,  the  win/win  strategy  for  negotiation 
was  integrated  into  two  key  game  design  elements 
identified  in  this  prototype.  First,  the  preparation  phase  of 
the  game  required  the  player  to  identify  the  negotiation 
partner’s  desired  outcomes  promoting  a  win/win 
framework  by  promoting  a  student’s  understanding  of  a 
negotiation  partners  needs  or  wants  (e.g.,  a  police  captain 
needs  helmets  and  flashlights  for  his  officers). 

The  second  aspect  of  the  game  reinforcing  this 
training  objective  was  identified  in  the  actual  negotiation 
phase  of  the  game.  If  the  student  learned  about  the  need 
for  helmets  and  flashlights,  s/he  was  expected  to  try  to 
acquire  those  resources,  then  offer  them  during  the 
negotiation.  Without  these  to  offer,  the  student  would  not 
achieve  a  win/win  outcome  and  the  chances  of 
successfully  negotiating  would  be  reduced. 

3.  Implement  Computer  Prototype.  Using  the  paper 
prototype  as  a  development  plan,  the  transition  to  a 
computer  prototype  was  initiated.  This  included  design  of 
the  user  interface,  implementation  of  the  game  mechanics, 
implementation  of  the  authoring  environment,  and 
encoding  of  the  initial  scenario  content.  As  this  occurred. 


further  SME  feedback  was  incorporated  to  validate  game 
mechanics  and  content. 

When  working  versions  of  the  system  were  available, 
playability  testing  was  conducted  with  training  instructors 
and  other  Army  personnel.  Playability  testing  provided 
an  important  iterative  role  in  the  development  process.  By 
putting  pre-alpha  game  versions  in  front  of  end-users, 
feedback  could  be  collected  about  game  mechanics,  in¬ 
game  content,  and  realism  of  Army  tactics,  techniques, 
and  procedures  (TTPs)  modeled  in  the  computer 
environment. 

An  authoring  tool  was  developed  along  with  the 
initial  prototype.  It  promotes  a  pedagogical  approach  to 
content  development  by  situating  training  objectives  as 
the  fundamental  component  of  a  new  scenario,  requiring 
that  they  be  created  first.  As  game  content  is  developed, 
the  learning  objectives  are  linked  with  relevant  training 
objectives  so  that  the  coach  can  perform  assessments  and 
the  reflective  tutor  can  conduct  AARs.  The  win/win 
training  objective  is  connected  to  game  actions  such  as 
telling  your  partner  that  the  “U.S.  wants  to  cooperate  and 
work  with  the  Iraqi  police”  (an  example  of  the  subtask  of 
developing  relationships). 

4.  Refine  Training  Objectives,  Conditions  and 
Standards.  As  described  above,  the  training  objectives 
and  their  related  sub-tasks  were  refined,  and  linked  to 
game  actions  as  a  means  of  "‘scoring”  and  continuous 
assessment.  The  ITS  in  the  game  monitors  play  on  a  turn- 
by-turn  basis  and  keeps  a  record  of  the  learner’s  successes 
and  failures.  A  second  benefit  of  explicitly  linking 
training  objectives  to  game  content  is  that  it  provides  an 
indirect  confirmation  of  the  game  content.  That  is,  if  it  is 
found  that  certain  game  actions  do  not  seem  to  support 
any  training  objective,  or  that  some  training  objectives  are 
“orphaned”,  then  it  is  clear  that  revision  is  needed. 

This  organization  of  training  domain  content  allowed 
for  the  identification  of  commonalities  among  tasks  that 
were  considered  appropriate,  tasks  that  were  considered 
inappropriate,  and  the  conditions  that  influenced  the 
appropriateness  of  these  tasks.  This  structuring  included 
both  what  should  be  done  and  what  should  be  avoided, 
which  allowed  the  ITS  to  identify  both  correct  and 
incorrect  responses  and  the  conditions  that  influence  the 
appropriateness  of  actions. 

For  example,  if  the  student  selects  the  action  to  tell 
his/her  partner  the  U.S.  will  cooperate  with  local  police  at 
the  appropriate  time  -  during  a  business  phase  -  the  coach 
will  recognize  this  as  a  positive  action,  and  if  the  positive 
feedback  is  “turned  on,”  deliver  the  message  “This  action 
builds  trust  and  rapport,”  which  reinforces  the  training 
objective  of  creating  win-win  situations.  If  the  student 
needs  a  hint,  the  message  would  be  “How  can  you 
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collaboratively  resolve  this  conflict?”  If  the  action  is 
taken  during  a  “social”  period,  it  will  be  considered  an 
error.  A  different  training  objective  in  the  game  is  to 
follow  the  social  lead  of  the  host  -  if  the  player  attempts 
to  talk  about  business  before  being  cued  to  do  so,  there 
are  similar  (negative)  feedback  messages  for  that  case. 
The  reflective  tutor  can  devote  more  time  to  discussion 
since  it  runs  after  the  meeting  when  it  won’t  break  the 
flow,  so  its  utterances  include  deeper  explanations  for 
why  actions  are  good  or  bad  and  may  discuss  possible 
alternative  courses  of  action. 

5.  Develop  Training  Support  Material.  Lastly,  a  set 
of  learning  material  and  instructions  for  training 
developers,  instructors  and  students  was  produced  to 
allow  for  turn-key  employment  of  the  learning 
application,  either  as  part  of  an  existing  Program  of 
Instruction  (POI)  or  as  a  stand  alone  learning  module. 

This  training  support  material  includes  introductory 
information  on  the  training  domain  so  that  trainees  can 
learn  the  information  that  they  will  practice  while  using 
the  ELECT-BiLAT  training  system.  Also  included  in  the 
support  material  are  directions  for  instructors  on  how  to 
use  the  game  interface,  work  through  the  phases  of  the 
game  and  ideas  on  how  to  use  the  game  to  promote 
discussion  and  learning  in  a  class.  Much  like  the  reflective 
tutor,  these  training  materials  help  tie  the  training 
objectives  to  the  game  content.  In  the  case  of  the  win/win 
example,  the  materials  provide  background  and  examples 
to  show  how  an  instructor  might  discuss  successful 
negotiation  tactics  in  the  context  of  the  game. 

8.  CONCLUSION 

In  the  past,  some  of  the  difficulties  with  game-based 
training  included  a  lack  of  appropriate  instructional 
techniques,  unrealistic  or  inappropriate  training  scenarios, 
and  poor  communication  between  instructors/SMEs  and 
game  developers  (Belanich,  Mullin,  &  Dressel,  2004; 
Hays,  2005).  In  the  ELECT-BiLAT  project  these 
concerns  were  addressed  by  developing  a  system  based  on 
sound  pedagogical  principles,  creating  training  scenarios 
drawing  on  and  validated  by  the  instructors  and  SMEs, 
and  applying  an  iterative  development  process  with 
frequent  interaction  between  instructor/SMEs  and  the 
training  system  developers. 

As  part  of  the  goal  to  create  a  rapid  development 
framework,  work  is  being  done  to  make  in-game  content 
and  functions  authorable.  These  tools  are  being  targeted  at 
a  number  of  different  users.  One  user  is  an  instructor  who 
could  use  desktop  and  web-based  tools  to  manipulate  and 
generate  content  without  the  support  of  a  programmer. 
Another  user  is  the  training  developer  who  will  build  and 
maintain  the  applications.  And,  finally,  some  of  the  tools 


and  methods  being  developed  such  as  the  SmartBody 
animation  engine  aim  to  reduce  the  authoring  needs  of  the 
content  developers. 

The  development  framework  used  for  ELECT 
BiLAT  has  already  yielded  much  in  the  way  of  defining  a 
deliberate  approach  for  the  development  of  game-based 
learning  applications.  The  use  of  training  objectives, 
based  on  a  task  analysis  of  the  training  domain,  as  the 
foundation  for  developing  pedagogically  sound  training 
provides  implicit  validation  of  game  content,  promotes 
relevant  tutorial  feedback,  and  acts  as  the  basis  for 
automated  assessment.  This  structure  allows  game-based 
training  to  be  developed  with  clear  learning  goals  and  a 
means  to  reach  those  goals.  Such  training  systems  will 
provide  our  Future  Force  with  the  skills  needed  to  be 
successful. 
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